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Program Description 1 2 

Fast ForWord® is a computer-based reading program intended 
to help students develop and strengthen the cognitive skills 
necessary for successful reading and learning. The program, 
which is designed to be used 30 to 100 minutes a day, five 
days a week, for 4 to 16 weeks, includes two components. The 
first component, the Fast ForWord® Language 2 and Literacy 3 4 
series, aims to build cognitive skills such as memory, attention, 
processing, and sequencing, as well as language and reading 
skills, including listening accuracy, phonological awareness, 

the Fast ForWord® to Reading 4 series (also known as the 
Fast ForWord® Reading series), aims to increase processing 
efficiency and further improve reading skills such as sound-letter 
associations, phonological awareness, word recognition, knowledge 
of English language conventions, vocabulary, and comprehension. 
The program, developed by scientists with expertise in the areas 
of brain plasticity, cognitive development, and reading instruc- 
tion, is designed to adapt the nature and difficulty of the content 
based on individual students’ responses. 


and knowledge of language structures. The second component, 


1. The descriptive information for this program was obtained from a publicly available source: the program’s website (http://www.scilearn.com, down- 
loaded July 2009), as well as information provided to the WWC by the developer. The WWC requests developers to review the program description 
sections for accuracy from their perspective. Further verification of the accuracy of the descriptive information for this program is beyond the scope of 
this review. The literature search reflects documents publicly available through December 2008. 

2. The Fast ForWord® Language series, designed for elementary school students, includes three products: (1) Fast ForWord ® Language Basics , which 
focuses on sound sequencing, fine motor skills, hand-eye coordination, pattern recognition, and color-shape recognition; (2) Fast ForWord ® Language, 
which focuses on listening accuracy, phonological awareness, and language structures; and (3) Fast ForWord ® Language to Reading , which focuses on 
the link between spoken and written language. 

3. The Fast ForWord ® Literacy series, designed for secondary school students and adults, includes two products: (1) Fast ForWord ® Literacy, which 
focuses on listening accuracy, phonological awareness, and language structures; and (2) Fast ForWord ® Literacy Advanced, which focuses on pro- 
cessing efficiency, memory, concentration, comprehension, and sequencing. Students in at least two of the studies included in this review used Fast 
ForWord® Middle and High School, which was discontinued and replaced by the Fast ForWord ® Literacy series. 

4. The Fast ForWord ® Reading series, designed for students at all reading levels, includes six products. Fast ForWord ® Reading Prep focuses on letter 
recognition, phonological awareness, and letter-sound associations. Fast ForWord® Reading Levels 1, 2, 3, 4, and 5 focus on a variety of skills, depend- 
ing on the level. For example, level 1 focuses on early reading skills such as phonemic awareness, early decoding skills, vocabulary knowledge, and 
motivation for reading, and level 5 focuses on skills suitable for more advanced readers in upper elementary, middle, or high school, such as reading 
comprehension and vocabulary skills. 
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Research 5 


Two studies of Fast ForWord® that fall within the scope of the 
Adolescent Literacy review protocol meet What Works Clearing- 
house (WWC) evidence standards, and six studies meet WWC 
evidence standards with reservations. The eight studies included 
about 2,000 students, ranging in age from 5 to 17, who attended 
elementary, middle, and high schools 5 6 in Indiana, Maryland, 


North Carolina, Ohio, Pennsylvania, Virginia, an urban district in 
the northeastern United States, and Australia. 7 

Based on these eight studies, the WWC considers the extent 
of evidence for Fast ForWord ® on adolescent learners to be 
small for the alphabetics and reading fluency domains and 
medium to large for the comprehension and general literacy 
achievement domains. 


Effectiveness Fast ForWord ® was found to have no discernible effects on the alphabetics and general literacy achievement domains, and poten- 
tially positive effects on the reading fluency and comprehension domains for adolescent learners. 


Rating of effectiveness 
Improvement index 8 9 


General literacy 

Alphabetics Reading fluency Comprehension achievement 

No discernible effects Potentially positive Potentially positive No discernible effects 

effects effects 


Average: +2 percentile 
points 


+17 percentile points 


Average: +8 percentile 
points 


Average: +3 percentile 
points 


Range: -8 to +9 na 

percentile points 


Range: -6 to +15 Range: -1 to +9 

percentile points percentile points 


na = not applicable 


Additional program 
information 


5. The studies in this report were reviewed using WWC Evidence Standards, Version 1.0 (see the WWC Standards), as described in protocol Version 1.0. 

6. The Adolescent Literacy topic area reviews studies of interventions administered to students in grades 4-12 (or 9-18 years of age). For studies that 
include samples of students that span both the Adolescent Literacy (grades 4-12) and Beginning Reading (grades K-3) topic areas and cannot be disag- 
gregated by grade level, the Adolescent Literacy topic area also reviews any studies that include 5th-grade students or higher. For example, this report 
includes a combined sample of students from grades 3-6 (Rouse & Krueger, 2004), grades 2-8 (Scientific Learning Corporation, 2004b), and students 
aged 5-14 years (Scientific Learning Corporation, 2007a). 

7. The evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes available. 

8. These numbers show the average and range of student-level improvement indices for all findings across the studies. 

9. The year that Fast ForWord ® Literacy entered the market was unavailable. 


Developer and contact 

Fast ForWord ® was designed by university-based scientists, 

Drs. Merzenich, Jenkins, Tallal, Miller, and Mann, all of whom 
have expertise in the areas of brain plasticity, cognitive develop- 
ment, and reading instruction. Fast ForWord® is produced and 
distributed by the Scientific Learning Corporation, 300 Frank 
H. Ogawa Plaza, Suite 600, Oakland, CA 94612-2040. Email: 
customerservices@scilearn.com. Web: http://www.scilearn.com. 
Telephone: (888) 665-9707. Fax: (510) 444-3580. The program 


can be purchased from local Fast ForWord® providers who are 
listed in the searchable database on the Scientific Learning 
Corporation website. 

Scope of use 

Fast ForWord ® products entered the market with Fast ForWord® 
Language in 1997 and Fast ForWord ® to Reading (also known 
as Fast ForWord® Reading) in 2000.® Fast ForWord® products 
have been used by students struggling with reading, language, 
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Additional program and learning problems, as well as with the general K-12 student 
information population across the United States. Overall, Fast ForWord® has 
(continued) been used by more than 570,000 students in more than 3,700 
schools nationwide. 

Teaching 

The Fast ForWord® Language, Fast ForWord ® to Reading, and 
Fast ForWord® Literacy computer software uses exercises that 
aim to develop the cognitive processes necessary for reading. 
Fast ForWord® Language intends to build cognitive skills of 
memory, attention, processing, and sequencing, as well as 
language and reading skills, such as listening accuracy, pho- 
nological awareness, and language structures. Fast ForWord® 
to Reading aims to further improve cognitive and reading 
skills through exercises focused on sound-letter associations, 
phonological awareness, word recognition, knowledge of English 
language conventions, vocabulary, and comprehension. Fast 


ForWord® Literacy intends to improve students’ skills in the 
areas of listening accuracy, phonological awareness, language 
structures, processing efficiency, memory, concentration, 
comprehension, and sequencing. As students listen through 
headphones and respond using the mouse, the software adapts 
to individual students’ responses, adjusting the content and dif- 
ficulty of items presented so that the student responds correctly 
approximately 80% of the time. The developer suggests multiple 
options for using the program, ranging from 30 minutes a day, 
five days a week, for 12 to 16 weeks, to 90 to 100 minutes a day, 
five days a week, for 4 to 8 weeks. All children start at the same 
basic level and progress individually as they attain proficiency. 

Cost 

A single license for Fast ForWord® Language is $900, with 
discounts available for multiple licenses. Each license for Fast 
ForWord® to Reading is $500, with no quantity discount. 


Research A total of 305 studies reviewed by the WWC investigated the 
effects of Fast ForWord® on adolescent learners. Two studies 
(Rouse & Krueger, 2004; Scientific Learning Corporation, 2007a) 
are randomized controlled trials that meet WWC evidence 
standards. Six studies (Beattie, 2000; Borman & Benson, 2006; 
Overbay & Baenen, 2002; Scientific Learning Corporation 
2004a, 2004b, 2007b) are randomized controlled trials or quasi- 
experimental designs that meet WWC evidence standards with 
reservations. The remaining 297 studies do not meet either WWC 
evidence standards or eligibility screens. 

Meets evidence standards 

Rouse and Krueger (2004) conducted a randomized controlled 
trial of students in grades 3-6 in an urban district in the north- 
eastern United States. Students scoring in the bottom 20% on 
the state’s standardized reading test were randomly assigned 
within each grade and school to either the treatment group or 
the control group. The WWC based its effectiveness ratings on 
findings from comparisons of 237 students who received Fast 


ForWord® as a supplemental targeted pullout program during 
the regular school day and 217 control students who received 
regular reading instruction. The study reported students’ out- 
comes after six to eight weeks of program implementation. 

Scientific Learning Corporation (2007a) conducted a 
randomized controlled trial of 5- to 14-year-old students from 
four primary schools in the Perth metropolitan area in Western 
Australia. Students who had difficulties with language, literacy, 
auditory processing, attention, and/or behaviors were ran- 
domly assigned to the treatment and control groups. The WWC 
based its effectiveness rating on findings from comparisons 
of 68 students who received Fast ForWord® and 69 control 
group students who received regular classroom instruction. 

The study reported students’ outcomes after three months of 
program implementation. 

Meets evidence standards with reservations 

Beattie (2000) conducted a randomized controlled trial of 
middle and high school students in suburban northern Virginia. 
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Research (continued) 


Students with language deficits who ranged in age from 11 to 
16 were randomly assigned by computer-generated procedures 
to one of five groups (Appendix 1.1 provides more details about 
these groups). The WWC based its effectiveness rating on 
findings from comparisons of 12 students who received Fast 
ForWord® and 12 control group students who received regular 
reading instruction. Although these analytic samples were shown 
to be equivalent at baseline, overall attrition of the study sample 
led to the study’s rating of meets standards with reservations. 
The study reported students’ outcomes after two months of 
program implementation. 

Borman and Benson (2006) conducted a randomized 
controlled trial of 7th-grade students attending seven middle 
schools in Baltimore, Maryland. Students scoring below the 50th 
percentile on a district-administered reading test were randomly 
assigned within schools to either the treatment or the control 
group. Ninety students received the Fast ForWord® program 
as a supplemental targeted pullout program during the regular 
school day. Although post-attrition analytic samples were shown 
to be equivalent at baseline, overall and differential attrition of 
the study sample led to the study’s rating of meets standards 
with reservations. The 98 students in the control group received 
nonliteracy instruction or participated in special activities and 
classes, such as art and gym, for their supplemental instruction. 
The study reported students’ outcomes after two months of 
program implementation. 

Overbay and Baenen (2002) conducted a quasi-experimental 
study that examined the effect of Fast ForWord® on students 
from the Wake County Public School System in Raleigh, North 
Carolina. The students participating in Fast ForWord® were 
matched to students from schools that were not using Fast 
ForWord® based on demographic factors and reading pretest 
scores. The WWC based its effectiveness rating on findings from 
comparisons of 355 students from grades 4-8 who used Fast 
ForWord® and 355 comparison group students who did not. The 
study reported students’ outcomes after one academic year of 
program implementation. 


Scientific Learning Corporation (2004a) conducted a quasi- 
experimental study that examined the effect of Fast ForWord® 
on 4th-grade students in four schools in Springfield, Ohio. 
Students who did not pass the Ohio Proficiency Test in 2002 
constituted the study sample. The WWC based its effective- 
ness rating on findings from comparisons of 41 students who 
received Fast ForWord® and 50 comparison group students 
who attended schools that were not using Fast ForWord® and, 
like treatment group students, did not pass the Ohio Proficiency 
Test. The study reported students’ outcomes after one semester 
of program implementation. 

Scientific Learning Corporation (2004b) conducted a quasi- 
experimental study that examined the effect of Fast ForWord® on 
students from 16 public schools in Philadelphia, Pennsylvania. 
Students (primarily from 4th and 5th grades) were assigned 
to one of the three study groups. Group 1 received the Fast 
ForWord® intervention from September to November, group 
2 received the Fast ForWord® intervention from December to 
February, and group 3 served as the control. The WWC based 
its effectiveness rating on findings from comparisons of 125 
students in group 1 and 37 control group students, as well as 
comparisons of 131 students in group 2 and 37 control group 
students. The study reported students’ outcomes after three 
months of program implementation. 

Scientific Learning Corporation (2007b) conducted a quasi- 
experimental study that examined the effect of Fast ForWord® on 
students in grades 2-5 in Pendleton, Indiana. Students selected 
to receive the Fast ForWord ® intervention were individually 
matched by school personnel, using grade-level and reading test 
scores, to students not using Fast ForWord®. The WWC based 
its effectiveness rating on findings from comparisons of 35 
students in grades 4 and 5 who received Fast ForWord ® and 35 
comparison students who received the regular school curricu- 
lum. The study reported students’ outcomes after four months of 
program implementation. 
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Research (continued) Extent of evidence 

The WWC categorizes the extent of evidence in each domain 
as small or medium to large (see the WWC Procedures and 
Standards Handbook, Appendix G). The extent of evidence 
takes into account the number of studies and the total sample 


size across the studies that meet WWC evidence standards with 
or without reservations. 10 

The WWC considers the extent of evidence for Fast ForWord® 
to be small for the alphabetics and reading fluency domains, 
and medium to large for the comprehension and general literacy 
achievement domains for adolescent learners. 


Effectiveness Findings 

The WWC review of interventions for Adolescent Literacy 
addresses student outcomes in four domains: alphabetics, 
reading fluency, comprehension, and general literacy achieve- 
ment. The studies included in this report cover all four domains. 
Alphabetics includes five constructs: phonemic awareness, 
phonological awareness, letter knowledge, print awareness, 
and phonics. Comprehension includes two constructs: read- 
ing comprehension and vocabulary development. General 
literacy achievement includes two constructs: general reading 
achievement and other literacy achievement. The findings below 
present the authors’ estimates and WWC-calculated estimates 
of the size and the statistical significance of the effects of Fast 
ForWord® on adolescent learners. 11 

Alphabetics. Two studies reviewed findings in the alphabetics 
domain. Scientific Learning Corporation (2007a) did not find a 
statistically significant effect of Fast ForWord® on the Queen- 
sland University Inventory of Literacy (QUIL), nor was the effect 
large enough to be considered substantively important accord- 
ing to the WWC criteria (that is, an effect size of at least 0.25). 
Beattie (2000) did not find statistically significant effects of Fast 
ForWord® on the Letter-Word Identification, Word Attack, and 


Auditory Processing subtests of the Woodcock-Johnson tests 
of cognitive ability, or on the Wide Range Achievement Spelling 
subtest. The effects also were not large enough to be considered 
substantively important according to WWC criteria. 

For the alphabetics domain, both studies showed indetermi- 
nate effects. 

Reading fluency. Beattie (2000) did not find a statistically sig- 
nificant effect of Fast ForWord® on the Gray Oral Reading Test, 
but the effect was large enough to be considered substantively 
important according to WWC criteria. 

Comprehension. Six studies reviewed findings in the com- 
prehension domain. Beattie (2000) did not find a statistically 
significant effect of Fast ForWord ® on the Woodcock-Johnson 
Passage Comprehension subtest, but the effect was large 
enough to be considered substantively important according to 
WWC criteria. Borman and Benson (2006) did not find a statisti- 
cally significant effect of Fast ForWord® on the Terra Nova Read- 
ing test, and the effect was not large enough to be considered 
substantively important according to WWC criteria. Overbay and 
Baenen (2002) did not find a statistically significant effect of Fast 
ForWord® on the North Carolina End of Grade Reading Test, and 
the effect was not large enough to be considered substantively 


10. The extent of evidence categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the 
number and size of studies. Additional factors associated with a related concept— external validity, such as the students’ demographics and the types 
of settings in which studies took place— are not taken into account for the categorization. Information about how the extent of evidence rating was 
determined for Fast ForWord ® is in Appendix A6. 

11. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within class- 
rooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate 
the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, 
Appendix D for multiple comparisons. In all studies except Borman and Benson (2002), Scientific Learning Corporation (2004a, 2004b), and Overbay and 
Baenen (2002), a correction for multiple comparisons was needed, so the significance levels may differ from those reported in the original studies. 
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Effectiveness (continued) 


important according to the WWC criteria. Scientific Learning 
Corporation (2004a) did not find a statistically significant effect 
of Fast ForWord® on the Ohio Proficiency Test Reading score, 
but the effect was large enough to be considered substantively 
important according to WWC criteria. Scientific Learning Cor- 
poration (2004b) found, and the WWC confirmed, a statistically 
significant effect of Fast ForWord ® on the Gates-MacGinitie 
Reading Test. Scientific Learning Corporation (2007b) did not 
find a statistically significant effect of Fast ForWord® on the 
Reading Measure of Academic Progress, and the effect was not 
large enough to be considered substantively important accord- 
ing to WWC criteria. 

For the comprehension domain, one study showed statisti- 
cally significant positive effects, two studies showed substan- 
tively important positive effects, and three studies showed 
indeterminate effects. 

General literacy achievement. Five studies reviewed find- 
ings in the general literacy achievement domain. Rouse and 
Krueger (2004) did not find statistically significant effects of Fast 
ForWord® on the Clinical Evaluation of Language Fundamentals, 
Success for All assessment, and a state standardized reading 
test, and none of the effects were large enough to be considered 
substantively important according to WWC criteria. Scientific 
Learning Corporation (2007a) did not find statistically significant 
effects of Fast ForWord® on the Clinical Evaluation of Language 
Fundamentals Receptive and Expressive subtests, and neither 


of the effects was large enough to be considered substantively 
important according to WWC criteria. Borman and Benson 
(2006) did not find a statistically significant effect of Fast ForWord ® 
on the Terra Nova Language test, and the effect was not large 
enough to be considered substantively important according 
to WWC criteria. Beattie (2000) did not find a statistically 
significant effect of Fast ForWord® on the Clinical Evaluation of 
Language Fundamentals, nor was the effect large enough to be 
considered substantively important according to WWC criteria. 
Scientific Learning Corporation (2007b) did not find a statistically 
significant effect of Fast ForWord® on the Language Measure of 
Academic Progress, and the effect was not large enough to be 
considered substantively important according to WWC criteria. 

For the general literacy achievement domain, all five studies 
showed indeterminate effects. 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as positive, potentially positive, mixed, no discernible 
effects, potentially negative, or negative. The rating of effective- 
ness takes into account four factors: the quality of the research 
design, the statistical significance of the findings, the size of 
the difference between participants in the intervention and the 
comparison conditions, and the consistency in findings across 
studies (see the WWC Procedures and Standards Handbook, 
Appendix E). 


The WWC found 
Fast ForWord® to have 
no discernible effects 
for the alphabetics 
and general literacy 
achievement domains and 
potentially positive effects 
for the reading fluency and 
comprehension domains 
for adolescent learners 


Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and an 
average improvement index across studies (see WWC Proce- 
dures and Standards Handbook, Appendix F). The improvement 
index represents the difference between the percentile rank 
of the average student in the intervention condition and the 
percentile rank of the average student in the comparison condi- 
tion. Unlike the rating of effectiveness, the improvement index is 


entirely based on the size of the effect, regardless of the statisti- 
cal significance of the effect, the study design, or the analysis. 
The improvement index can take on values between -50 and 
+50, with positive numbers denoting favorable results for the 
intervention group. 

The average improvement index for alphabetics is +2 
percentile points across two studies, with a range of -9 to +9 
percentile points across findings. The improvement index for 
reading fluency is +17 percentile points for a single finding from 
one study. The average improvement index for comprehension 
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The WWC found 
Fast ForWord® to have no 
discernible effects for the 
alphabetics and general 
literacy achievement domains 
and potentially positive effects 
for the reading fluency and 
comprehension domains for 
adolescent learners (continued) 
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Appendix 


Appendix A1.1 Study characteristics: Rouse & Krueger, 2004 


Characteristic 

Description 

Study citation 

Rouse, C. E., & Krueger, A. B. (2004). Putting computerized instruction to the test: A randomized evaluation of a “scientifically based” reading program. Economics of 
Education Review, 23(4), 323-338. 

Participants 

Groups were formed through a multistep process. Authors first identified an eligible population of students from four schools within one urban school district, focusing on 
third- to sixth-grade students who scored in the bottom 20% on the state's standardized reading test administered in the 2001-02 school year. Consent letters were sent 
to these students' parents. Principals in the schools were asked to identify students who could not sit through the daily 90- to 100-minute use of Fast ForWorcP , those who 
had transferred to another school, and those students who might otherwise be unavailable (family away on long trip, for example). The remaining students were randomly 
assigned to either the treatment or control group, within each grade and school. In all, 237 students in the Fast ForWorcP group and 217 students in the comparison group 
were included in the analysis sample. 

Setting 

The study took place in four schools in an urban district in the northeastern United States. Forty percent of the district's students were African-American and more than 50% 
were Hispanic. Almost 70% of students in the district qualified for the free or reduced-price lunch program, and 56% of the district’s students spoke a language other than 
English at home. The authors describe test scores in these schools as well below average and note that schools in the district adopted a whole-school reform, Success for All. 

Intervention 

Fast ForWorcP was primarily an add-on to regular reading instruction. In three schools, students in the treatment condition were pulled out of their regular classroom 
instruction for 90-100 minutes of computerized Fast ForWorcP instruction per day and, in one school, they used Fast ForWorcP for that same amount of time before 
or after school. Each school had to find a way to fit the use of Fast ForWorcP into its unique schedule. In no case were students taken out of Success for All. The study 
reported students’ outcomes after six to eight weeks of program implementation. 

Comparison 

The control group continued to receive the standard curriculum being used in district schools. Because the Fast ForWorcP students used Fast ForWorcP either during subjects 
such as math, science, language arts, special subjects (such as art, music, or gym), or homeroom, or — in the case of one school — before or after school, the counterfactual 
condition for the control group students was mixed. 

Primary outcomes 
and measurement 

For both the pretest and posttest, the authors administered the Success for All assessment, the Clinical Evaluation of Language Fundamentals-Third Edition (the receptive 
portion and the Listening to Paragraph supplemental test), and a state standardized reading test (the authors did not indicate which state). For a more detailed description of 
test outcome measures, see Appendix A2.4. 

Staff/teacher training 

Fast ForWorcP staff provided training for Fast ForWorcP instructors (those interacting with students) at the beginning of the study. Phone support was also provided for the 
duration of the study. Detailed information on the training of instructors was not provided. 
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Appendix A1.2 Study characteristics: Scientific Learning Corporation, 2007a 


Characteristic 

Description 

Study citation 

Scientific Learning Corporation. (2007a). Students in Western Australia improve language and literacy skills: Educator’s briefing. Oakland, CA: Author. 

Participants 

Students between ages 5 and 14 identified by classroom teachers as having difficulties in language, literacy, auditory processing, attention, and/or behavior were randomly 
assigned to immediate or delayed treatment conditions, with 72 students in each group. The intervention group that received Fast ForWorcP either between February and April 
or May and July of 2006 was compared to the group of students who had not received Fast ForWorcP as of April 2006. In all, 68 students in the Fast ForWorcP group and 69 
students in the comparison group were included in the analysis sample. 

Setting 

The study took place at four primary schools in the Perth metropolitan area in Western Australia. 

Intervention 

Fast ForWorcP participation was scheduled during class time for most students, generally in place of their language-arts lesson. A few students participated before school and 
during recess and/or lunch breaks. All Fast ForWorcP sessions were monitored by trained parent volunteers under the supervision of the school's Fast ForWorcP coordinator. 
Participants in the Fast ForWorcP group used (1) the 50-minute Fast ForWorcP Language protocol or the 48-minute Fast ForWorcP Middle and High School protocol and (2) the 
50-minute Fast ForWorcP Language to Reading protocol. These protocols called for participants to use Fast ForWorcP each day, five days a week, for 8 to 12 weeks. The study 
reported students’ outcomes after three months of program implementation. 

Comparison 

The counterfactual in this study is regular classroom instruction. The comparison group used Fast ForWorcP on a delayed schedule, either between May and July or July 
and September 2006. 

Primary outcomes 
and measurement 

All tests were administered by speech pathology and occupational therapy students who were trained in the assessment process by qualified speech pathologists. Study 
students’ skills were measured both before and after use of the intervention. Alphabetic skills were measured by the Queensland University Inventory of Literacy (QUIL), 
whereas students' skills in comprehension were measured by the Clinical Evaluation of Language Fundamentals (CELF)-Fourth Edition. For a more detailed description of 
these outcome measures, see Appendices A2.1 and A2.4. 

Staff/teacher training 

Sonic Hearing, a private clinical practice with expertise in the Fast ForWorcP programs, provided training for the parent monitors and support for the Fast ForWorcP coordinator 
at each school. All Fast ForWorcP sessions were monitored by these trained parent volunteers, under the supervision of the school's Fast ForWorcP coordinator. In addition, 
the lab supervisors at the schools were trained in current and established findings on the neuroscience of how phonemic awareness and the acoustic properties of speech 
affect development of language and reading skills, information on the efficacy of the products, effective implementation techniques, and monitoring student progress. 
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Appendix A1.3 Study characteristics: Beattie, 2000 


Characteristic 

Description 

Study citation 

Beattie, K. K. (2000). The effects of intensive computer-based language intervention on language functioning and reading achievement in language-impaired adolescents 
(Doctoral dissertation, George Mason University, 2000). Dissertation Abstracts International, 61 { 08A), 194-3116. 

Participants 

Eighty-one 11- to 16-year-old students who scored in the bottom quartile on standardized reading or language tests were randomly assigned by computer-generated proce- 
dures to one of four intervention groups or to a control group in a two-step process. 1 The researchers first assigned 18 students to the two intervention groups that received 
a phase of SuccessMaker and Fast ForWord® and also concomitantly participated in a functional resonance imaging research project. Then, the remaining participants were 
randomly assigned across the five groups. To ensure an equal distribution among groups, fewer students were placed in the first two groups at the second step of randomiza- 
tion. For this review, the WWC reported results from 12 students in the Fast ForWorcP group who were compared to 12 students in the comparison group. 2 Although the overall 
attrition rate was higher than 20%, the post-attrition intervention and comparison groups were equivalent on the pretest achievement measures. 

Setting 

The study took place in two middle schools and one middle-high school located in the suburbs of a large metropolitan area in northern Virginia. 

Intervention 

Students worked on Fast ForWorcP for 90-94 minutes a day, five days a week. The intervention ended after each student completed 64-80 hours on the program. The study 
reported students’ outcomes after two months of program implementation. 

Comparison 

The control group received the standard instruction provided as part of the regular school curriculum. 

Primary outcomes 
and measurement 

For both pre- and posttests, the author administered the Gray Oral Reading Test, four subtests of the Woodcock-Johnson Psycho-Educational Battery (Letter-Word Identifica- 
tion, Word Attack, Passage Comprehension, and Auditory Processing), the Spelling subtest of the Wide Range Achievement Test, and the Receptive Language subtest of the 
Clinical Evaluation of Language Fundamentals, For a more detailed description of these outcome measures, see Appendices A2.1-A2.4, 

Staff/teacher training 

No information on training for the teachers and staff in this study was provided. To facilitate the use of Fast ForWortfl , computers were procured or updated to meet criteria for 
running FastForWorcP software. 


1. The first intervention group received two phases of Fast ForWord®; the second intervention group received two phases of SuccessMaker ; and the third and fourth intervention groups received a 
phase of Fast ForWord ® and a phase of SuccessMaker. 

2. The analysis samples for the Fast ForWord ® and SuccessMaker groups were not shown to be equivalent at baseline. Two other groups which combined Fast ForWord ® and SuccessMaker are 
not appropriate counterfactuals because the measures of effects cannot be attributed solely to the Fast ForWord ® program. 
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Appendix A1.4 Study characteristics: Borman & Benson, 2006 


Characteristic 

Description 

Study citation 

Borman, G. D., & Benson, J. (2006). Can brain research and computers improve literacy? A randomized field trial of the Fast ForWord® Language computer-based training 
program (WCER working paper no. 2006-5). Madison, Wl: University of Wisconsin-Madison, School of Education. 

Participants 1 

Students were eligible for the study if they scored below national norms on the total reading outcome for the district-administered Comprehensive Test of Basic Skills— Fifth 
Edition (CTBS/5) during the spring of 2000. These students also tended to have below-average outcomes on language skills. A total of 274 of these academically at-risk 
seventh-grade students took pretests (CTBS/5) in the spring of 2001 . Random assignment was conducted separately within each of seven schools. Of the initial intervention 
and comparison students, listwise deletion of students with missing pretest or posttest data was conducted. Additionally, 13 students (eight from the treatment group and five 
from the control group) were dropped from the sample because they were determined to be outliers based on a substantial drop from pre- to posttest. In all, 90 students in the 
Fast ForWorcP group and 98 students in the control group were included in the analysis sample (therefore, overall attrition was 31%). Although differential attrition between the 
treatment and control groups was 8%, the treatment and control groups were shown to be similar to each other at baseline. The groups primarily consisted of African-American 
(66.3% of both the intervention and comparison groups) and economically disadvantaged students (73.3% of the intervention group and 84.7% of the comparison group 
received free lunch). 

Setting 

The study took place in seven middle schools in the Baltimore City Public School System. 

Intervention 

In addition to their regular reading instruction, students randomly assigned to the intervention condition used the Fast ForWonfl Language software program in school resource 
rooms. The resource rooms served as a targeted pullout program offered during the regular school day to supplement the regular classroom literacy instruction. Students 
received the program 100 minutes a day, five days a week, for at least 20 days under the supervision of an Fast ForWord® -trained teacher. The study reported students’ 
outcomes after two months of program implementation. 

Comparison 

In addition to their regular reading instruction, comparison group students received nonliteracy instruction or participated in special activities and classes not related to literacy, 
such as art and gym. 

Primary outcomes 
and measurement 

The eligible outcomes are standardized (normal curve equivalent) CTBS/5 Terra Nova Language and Reading test scores. These tests were administered both before and after 
the intervention. For a more detailed description of test outcome measures, see Appendix A2.4. 

Staff/teacher training 

Before the start of the program, Scientific Learning provided training sessions for teachers operating the Fast ForWorcP programs at the schools. No detailed information about 
these training sessions was provided by the authors. 


1. In addition to the 188 students included in the analysis sample, the study also included 112 second-grade students who were excluded from the findings in this report because they did not fall in 
the grade range specified in the protocol. 
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Appendix A1.5 Study characteristics: Overbay & Baenen, 2002 


Characteristic 

Description 

Study citation 

Overbay, A., & Baenen, N. (2002). Fast ForWord® evaluation, 2002-03 (Evaluation and Research, report no. 03.24). Raleigh, NC: Wake County Public School System. 

Participants 

During the 2002-03 school year, 616 third- to eighth-grade students received the FastFoFWord® program. Of these, 426 were matched with students from non- Fast ForWord® 
schools based on race, limited English proficiency status, a special programs code, free and reduced-price lunch status, and reading pretest scores. The remaining 190 were 
missing either pre- or posttest scores and, therefore, were not included in the matching process. The analysis sample for this review included students in grades 4-8: 355 
students in the Fast ForWord® group and 355 in the comparison group. 1 Fast ForWord® was used in 10 elementary, middle, and high schools, and the comparison students 
were selected from schools that did not use Fast ForWord®. Additional findings reflecting students’ outcomes by grade can be found in Appendix A4. 

Setting 

The study took place in one school district (10 treatment schools) in Raleigh, North Carolina. 2 

Intervention 

During the school year, the intervention group used Fast ForWord ® Language, Fast ForWord® Language to Reading, and Fast ForWord® Reading. Most of the Fast ForWord® 
participants (91.4%) used Fast ForWorcP Language ; the majority (60%) used more than one level of the program. The 8.6% who did not use Fast ForWord® Language had 
completed it in 2001-02. 

Comparison 

The counterfactual in this study is regular classroom instruction. However, the study authors note that students in the comparison group may have been exposed to a variety of 
other programs or interventions that were not controlled for in this study. 

Primary outcomes 
and measurement 

For both pre- and posttests, the authors used the End of Grade Reading Subtest. For a more detailed description of this outcome measure, see Appendix A2.3. 

Staff/teacher training 

No information about teacher or staff training was provided. 


1. The study also presented data for students in grade 3, attending a total of six elementary schools, but these students do not fall within the age range of the WWC’s Adolescent Literacy reviews, 
so they are not included in this report. 

2. The number of control schools is not available. 
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Appendix A1.6 Study characteristics: Scientific Learning Corporation, 2004a 


Characteristic 

Description 

Study citation 

Scientific Learning Corporation. (2004a). Improved Ohio reading proficiency test scores by students in the Springfield City School District who used FastForWorcP products. 
MAPS for Learning: Educator Reports, 8(8), 1-6. 

Participants 

Fourth-grade students who did not pass the fall 2002 Ohio Proficiency Test from four Title 1 designated schools were eligible to participate in the study. Each elementary 
school established its own method of identifying treatment and comparison group students for the study. The comparison group was formed by selecting 50 students with test 
scores from both fall 2002 and spring 2003 who had no exposure to Fast ForWord ® products. 1 The intervention and comparison groups were shown to be equivalent on the 
Ohio Reading Proficiency Test pretest scores. In all, 41 students who used the FastForWorcP products and 50 students in the comparison group were included in the analysis 
sample. 

Setting 

The study took place in four elementary schools in the Springfield City School District in Ohio. 

Intervention 

The study used FastForWorcP Language, FastForWorcP Language to Reading, and Fast ForWorcP to Reading 3 products. The FastForWorcP Language protocol called for stu- 
dents to use the product for 100 minutes a day, five days a week, for four to eight weeks. The FastForWorcP Language to Reading and FastForWorcP to Reading 3 protocols 
called for use of the product for 90 minutes a day, five days a week, for four to eight weeks. Students included in the treatment group were required to have used Fast ForWorcP 
products for 20 or more days. Schools used different implementation models, with some schools having students use the products in the back of the classroom, and other 
schools sending students to computer labs that served between 7 and 24 students. The study reported students’ outcomes after one semester of program implementation. 

Comparison 

The study did not describe the comparison condition. Presumably, the comparison group received the regular school curriculum. 

Primary outcomes 
and measurement 

The Ohio Reading Proficiency Test (a statewide assessment) was administered in the year of the study, before and after the intervention. For a more detailed description of this 
outcome measure, see Appendix A2.3. 

Staff/teacher training 

At each participating school, educators were trained in current and established neuroscience findings on how phonemic awareness and the acoustic properties of speech 
affect development of language and reading skills, information on the efficacy of the products, methods for assessment of potential candidates for participation, the selection 
of appropriate measures for testing and evaluation, effective implementation techniques, approaches for using Progress Tracker reports to monitor student performance, and 
techniques for measuring the gains students have achieved after they have finished using FastForWorcP products. 


1. The study authors did not provide detailed information on how comparison group students were selected (stating that comparison group students were “pseudo-randomly” selected). 
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Appendix A1.7 Study characteristics: Scientific Learning Corporation, 2004b 


Characteristic 

Description 

Study citation 

Scientific Learning Corporation. (2004b). Improved reading achievement by students in the school district of Philadelphia who used Fast ForWord® products. MAPS for Learn- 
ing: Educator Reports, 8( 21), 1-6. 

Participants 

Three groups of students in grades 2 to 8 (mainly fourth and fifth graders) participated in Fast ForWord® supplemental instruction during the 2003-04 school year. Groups 1 
and 2 comprised the treatment group for this study. Group 1 used Fast ForWorcP between September and November, and group 2 used Fast ForWord® between December and 
February. Group 3 served as the comparison group (and used Fast ForWord® between March and May). The participating schools determined which students were placed in 
the three groups. Students were assessed in September and March. In all, 256 students in the Fast ForWord® treatment group and 37 students in the comparison group were 
included in the analysis sample. Additional findings reflecting students' outcomes by grade and intervention group (1 versus 2) can be found in Appendix A4. 

Setting 

The study took place in 16 schools in the Philadelphia School District in Pennsylvania. 

Intervention 

Students participating in the Fast ForWord® group used a variety of Fast ForWord® products. All students used either the Fast ForWord® Language or Fast ForWord® Middle and 
High School product for an average of 25 days. In addition, about half of the students used Fast ForWord® Language to Reading products (which are part of the Fast ForWord® 
Language series), and one-tenth of the students used Fast ForWord® Reading 3 products (which are part of the Fast ForWord® Reading series). Fast ForWord® was used as a 
supplement to the regular reading curriculum. The study reported students' outcomes after three months of program implementation. 

Comparison 

Before March 2004, comparison group students received their regular reading curriculum. 

Primary outcomes 
and measurement 

The eligible outcome on this study is the Gates— MacGinitie Reading Test, which was administered both before and after the intervention. For a more detailed description of this 
outcome measure, see Appendix A2.3. 

Staff/teacher training 

Teachers were trained in current and established findings on the neuroscience of how phonemic awareness and acoustic properties of speech impact development of language 
and reading skills; information on the efficacy of the products; methods for assessment of potential product participants; the selection of appropriate standardized language 
measures for testing and evaluation; effective implementation techniques; instruction on the product, Progress Tracker, and the reports generated by the product that allow 
educators and coaches to monitor student performance; and techniques for measuring the progress and gains students achieve after they have finished using the product. 
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Appendix A1.8 Study characteristics: Scientific Learning Corporation, 2007b 


Characteristic 

Description 

Study citation 

Scientific Learning Corporation. (2007b). Improved reading skills by students in the South Madison Community School Corporation who used Fast ForWord® products. MAPS 
for Learning: Educator Reports, 11 { 34), 1-7. 

Participants 

Two schools that used Fast ForWorcP during the spring of 2007 selected students in grades 2 to 5 for the study based on their scores on the Measures of Academic Progress 
(MAP) assessment. To form a comparison group, school personnel individually matched — by grade level and fall and winter scores from the MAP Reading subtest — 80 
students in the FastForWorcP group to 80 students not using Fast ForWorcP . The study sample included 78 treatment and 78 comparison students. The analysis sample for 
this review included students in grades 4 and 5: 35 students in the Fast ForWorrP group and 35 students in the comparison group. 

Setting 

This study took place in East Elementary and Maple Ridge Elementary in the South Madison Community School Corporation of Pendleton, Indiana. 

Intervention 

The intervention groups used Fast ForWortf Language and Fast ForWorcP Language to Reading products. The South Madison Community School Corporation chose to use 
the 50-minute FastForWorcP protocols, which called for students to use the product for 50 minutes a day, five days per week, for 6 to 10 weeks. The study reported students’ 
outcomes after three months of program implementation. 

Comparison 

The comparison group received the standard district reading curriculum. 

Primary outcomes 
and measurement 

The outcomes on this study are students' reading and language scores on the MAP assessment, which was administered both before and after the intervention was used for 
the study. For a more detailed description of this outcome measure, see Appendices A2.3-A2.4. 

Staff/teacher training 

Educators were trained in current and established neuroscience findings on how phonemic awareness and the acoustic properties of speech impact development of language 
and reading skills, information on the efficacy of the products, methods for assessing potential candidates for participation, the selection of appropriate measures for testing 
and evaluation, effective implementation techniques, approaches for using Progress Tracker reports to monitor student performance, and techniques for measuring the gains 
students have achieved after they have finished using FastForWorcP products. 
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Appendix A2.1 Outcome measures for the alphabetics domain 


Outcome measure 

Description 

Phonemic awareness 

Woodcock-Johnson 
Psycho-Educational 
Battery-Revised, Tests of 
Cognitive Ability (WJ-R COG) 
(Auditory Processing Cluster 
for Phonemic Awareness) 

This composite is a standardized measure of a student’s ability to identify patterns among speech-based auditory stimuli. The score on this composite is derived from scores 
on three subtests: (1) the Sound Blending subtest measures the ability to synthesize sequences of sounds into whole words, (2) the Incomplete Words subtest measures the 
ability to identify a word with missing sounds, and (3) the Sound Patterns subtest measures the ability to indicate whether pairs of computer-generated sound sequences are 
the same or different (as cited in Beattie, 2000). 

Phonics 

Woodcock-Johnson 
Psycho-Educational 
Battery-Revised, Tests of 
Achievement (WJ-R ACH) 
(Letter-Word Identification) 

This standardized subtest, which assesses students’ ability to identify words and letters, requires students to read aloud isolated letters and real words that range in frequency 
and difficulty (as cited in Beattie, 2000). 

Woodcock-Johnson 
Psycho-Educational 
Battery-Revised, Tests 
of Achievement (WJ-R 
ACH) (Word Attack) 

This standardized subtest measures phonemic decoding skills by asking students to read "pseudo” words (e.g., plurp, fronkett). Students are aware that the words are not real 
(as cited in Beattie, 2000). 

Wide Range Achievement 
Test-Third Edition (WRAT-3) 
(Spelling subtest) 

This standardized subtest is a paper-and-pencil assessment that measures students’ ability to write their names, as well as letters and words from dictation. The dictated 
letters and words followed either phonetically regular or irregular patterns (as cited in Beattie, 2000). 

Queensland University 
Inventory of Literacy (QUIL) 

The QUIL is a standardized clinical assessment tool for measuring the phonological awareness skills of school-age children as they pertain to literacy. Three of the 1 0 subtests 
were administered to all students: Nonword Spelling, Phoneme Segmentation, and Phoneme Manipulation. In addition, students in years 4-7 were administered the Spooner- 
isms subtest, which assesses students’ metalinguistic phoneme awareness (as cited in Scientific Learning Corporation, 2007a). 


Appendix A2.2 Outcome measures for the reading fluency domain 


Outcome measure 

Description 

Gray Oral Reading Test-Third 
Edition (GORT-3) 

In this standardized test, students are required to read orally a variety of graded passages to measure reading rate, word identification, and comprehension skills. The Passage 
subtest assesses a combination of rate and accuracy. The Comprehension subtest requires a student to respond to five multiple choice questions following each story. The 
Oral Reading Quotient reflects a total measure of a student’s oral reading performance and is calculated by combining the Passage and Comprehension scores (as cited in 
Beattie, 2000). 
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Appendix A2.3 Outcome measures for the comprehension domain 


Outcome measure 

Description 

Reading comprehension and vocabulary development 

Woodcock-Johnson 
Psycho-Educational 
Battery-Revised, Tests of 
Achievement (WJ-R ACH) 
(Passage Comprehension) 

In this standardized test. comDrehension is measured bv havina students fill in missina words in a short DaraaraDh (e.a.. “Woof.” said the . bitina the hand that 

fed it.) (as cited in Beattie, 2000). 

Ohio Proficiency Test 
(OPT), Reading subtest 

This statewide assessment is administered to students in 4th, 6th, and 9th grade. The Reading subtest includes multiple choice, short answer, and extended response 
questions across four subscales: constructing meaning from fiction, examining/extending meaning in fiction, constructing meaning from nonfiction, and examining/extending 
meaning in nonfiction. The subtest contains two or three fiction or poetry selections and two or three nonfiction selections, which may include pamphlets, instruction booklets, 
and newspaper and magazine articles. The selections total about 1 ,200 to 1 ,500 words. Students may be asked to summarize or retell a story, to interpret vocabulary, or to 
infer information. Students may also be asked to make predictions, to distinguish facts from opinions, or to fill in a chart or diagram with information from the selection. Word 
usage, grammar, spelling, and mechanics do not affect scoring, unless the student's ideas are not clear to the evaluator (as cited in Scientific Learning Corporation, 2004a). 

North Carolina End of 
Grade Test 

The North Carolina End of Grade test measures students' achievement of the goals and objectives specified in the 2004 North Carolina English Language Arts Standard 
Course of Study (Content Standards). Reading comprehension is assessed by having students read authentic selections and then answer questions directly related to the 
selections. Knowledge of vocabulary is assessed indirectly through application and understanding of terms within the context of selections and questions. The authentic 
selections in the reading tests are chosen to reflect reading for various purposes such as literary experience, gaining information, and performing a task (as cited in Overbay & 
Baenen, 2002). 

Gates-MacGinitie 
Reading Test (GMRT) 

The GMRT is used to assess a student's decoding, vocabulary, and passage comprehension skills. 1 The test has two components that independently assess reading vocabu- 
lary and comprehension skills. The Vocabulary subtest measures each student’s reading vocabulary by asking the student to choose one word or phrase that means most 
nearly the same as a presented word. The subtest contains 45 questions. The Comprehension subtest measures each student’s ability to read and understand different types 
of prose. The subtest contains 11 passages of various lengths and subjects, and 48 questions (as cited in Scientific Learning Corporation, 2004b). 

Measures of Academic 
Progress (MAP), Reading test 

Developed by the Northwest Evaluation Association (NWEA), the MAP are state-aligned computerized adaptive tests that reflect the instructional level of each student and 
measure growth over time. The MAP is appropriate for students in grades 2 through 10. The untimed assessment typically features between 40 and 50 items. The assess- 
ment is usually tailored to the specific needs of individual organizations, but all NWEA MAP assessments draw from the same item bank. The Reading test draws items 
from the following areas: word meaning (such as use of context clues; use of synonyms, antonyms, and homonyms; use of component structure; or interpretation of multiple 
meanings), literal comprehension (such as recalling details, interpreting directions, sequencing details, classifying facts, or identifying main ideas), interpretive comprehension 
(such as drawing inferences, recognizing cause and effect, predicting events, or summarizing and synthesizing), and evaluative comprehension (such as distinguishing fact and 
opinion, recognizing elements of persuasion, evaluating validity and point of view, evaluating conclusions, or detecting bias and assumptions) (as cited in Scientific Learning 
Corporation, 2007b). 

Comprehensive Test of 
Basic Skills (CTBS/5) Terra 
Nova Reading Composite 

This assessment combines selected-response items with constructed-response items that allow students to produce short and extended responses. The Reading composite 
score is the average of Reading Comprehension and Vocabulary subtest scores (as cited in TerraNova Prepublication Technical Manual, July 1996). The Reading Comprehen- 
sion subtest items focus on five objectives: (1) oral comprehension of passages read aloud, (2) basic understanding of literal meanings of passages, (3) analyzing text, (4) 
evaluating and extending meaning, and (5) identifying reading strategies. The Vocabulary subtest focuses on three objectives: (1) understanding word meaning, (2) identifying 
multi-meaning words, and (3) inferring words in context (as cited in Borman & Benson, 2006). 


1. At levels D (4th grade) and up, either subtest or the combination of both subtests falls into the comprehension domain. At levels A, B, and C (grades 1, 2, and 3), the vocabulary measure, which 
taps decoding skills rather than word meanings, would fall in the alphabetics domain. For the Scientific Learning Corporation (2004b) study, which included students from grades 2-8, the WWC 
classified the Gates-MacGinitie Reading Test as a comprehension measure, as the majority of study participants came from grades 4 and 5 (levels D and up). 
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Appendix A2.4 Outcome measures for the general literacy achievement domain 


Outcome measure 

Description 

General reading achievement 


Comprehensive Test of 
Basic Skills (CTBS/5) Terra 
Nova Language Composite 

This assessment combines selected-response items with constructed-response items that allow students to produce short and extended responses. The Language Com- 
posite score is the average of scores on the Language and Language Mechanics subtests (as cited in TerraNova Prepublication Technical Manual, July 1996). The Language 
subtest covers four objectives: (1) introduction to print, (2) understanding sentence structure, (3) writing strategies, and (4) editing skills. The Language Mechanics subtest 
focuses on three objectives: (1) appropriate construction of sentences, phrases, and clauses; (2) appropriate writing conventions; and (3) editing skills (as cited in Borman & 
Benson, 2006). 

Success for All assessment 

The Success for All assessments (which are administered every 6-8 weeks) are a set of reading assessments closely aligned with the Success for All curriculum. The total 
score of the assessments reflects (1) students’ scores on a paper-and-pencil assessment and (2) a more subjective assessment (by the evaluator) of the student’s class work 
during the time period. For example, the subjective assessment might evaluate how well children understand the learning objective, how their writing has progressed, and how 
well they comprehend what is read to them. Therefore, the total score not only reflects students' reading and writing achievement, but it can also reflect educational behaviors 
and habits (e.g., note taking, direction following, attention and focus). The version of the paper-and-pencil assessment administered to students depends on students' ability 
level and language proficiency. The assessments are designed to closely match the individual state’s assessment in both content and format (as cited in Rouse & Krueger, 
2004). 

State Standardized 
Reading Test 

This is the state’s criterion-referenced standardized test (the study authors did not specify which state). The exam is designed to be aligned with the curriculum standards of 
the state as well as to parallel critical aspects of the National Assessment of Educational Progress (NAEP). The state administers tests in reading, math, and writing annually 
(as cited in Rouse & Krueger, 2004). 

Clinical Evaluation of 
Language Fundamentals- 
Third Edition (CELF-3), 
Receptive Language 

This standardized assessment measures a student’s ability to interpret and execute commands of increasing complexity and to understand relationships between words 
and categories. It addresses sentence structure, concepts and directions, and word classes (as cited in Beattie, 2000). The Receptive Language portion of the assessment 
includes five components: (1) sentence structure, in which students point to one of four pictures in response to an orally presented stimulus; (2) concepts and directions, in 
which students identify pictures of geometric shapes in response to orally presented direction; (3) semantic relations, in which students listen to four facts and then select two 
of four visually presented options; (4) word classes, in which students select two out of three or four orally presented words that go together; and (5) recalling sentences, in 
which students imitate an orally presented sentence (as cited in Rouse & Krueger, 2004). 

CELF-4 — Australian 
Standard Edition, 
Receptive Language 

CELF-4 is a standardized test widely used to measure a student’s overall oral language ability. The Receptive Language index is a cumulative measure of students’ perfor- 
mance on subtests designed to best probe receptive aspects of language including comprehension and listening. The subtests cover topics such as Concepts & Following 
Directions, Word Classes, and Sentence Structure (as cited in Scientific Learning Corporation, 2007a). 

CELF-4 — Australian 
Standard Edition, 
Expressive Language 

CELF-4 is a standardized test widely used to measure a student’s overall oral language ability. The Expressive Language index is a cumulative measure of students' perfor- 
mance on subtests that probe expressive aspects of language including oral language expression. The subtests cover topics such as Word Structure, Recalling Sentences, and 
Formulated Sentences (as cited in Scientific Learning Corporation, 2007a). 

Measures of Academic 
Progress (MAP), 
Language Test 

Developed by the Northwest Evaluation Association, the MAP are state-aligned computerized adaptive tests that reflect the instructional level of each student and measure 
growth over time. The MAP is appropriate for students in grades 2 through 10. The untimed assessment typically features between 40 and 50 items. The assessment is 
usually tailored to the specific needs of individual organizations, but all NWEA MAP assessments draw from the same item bank. The Language Test draws items from the 
following areas: writing process, composition structure, grammar/usage, punctuation, and capitalization (as cited in Scientific Learning Corporation, 2007b). 
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Appendix A3.1 Summary of study findings for all domains 1 




Domain 



Alphabetics 

Reading fluency 


Comprehension 

General literacy achievement 


Rouse & Krueger (2004) 

nr 

Meets WWC evidence standards 

nr 

nr 

ind 

Scientific Learning Corporation (2007a) 

ind 

nr 

nr 

ind 



Meets WWC evidence standards with reservations 



Beattie (2000) 

ind 

(+) 

(+) 

ind 

Borman & Benson (2006) 

nr 

nr 

ind 

ind 

Overbay & Baenen (2002) 

nr 

nr 

ind 

nr 

Scientific Learning Corporation (2004a) 

nr 

nr 

(+) 

nr 

Scientific Learning Corporation (2004b) 

nr 

nr 

+ 

nr 

Scientific Learning Corporation (2007b) 

nr 

nr 

ind 

ind 

Rating of effectiveness 

No discernible effects 

Potentially positive effects Potentially positive effects 

No discernible effects 


nr = no reported outcomes under this domain 
+ = study finding was positive and statistically significant 

(+) = study finding was positive and substantively important, but not statistically significant 

ind = study finding was indeterminate; that is, neither substantively important nor statistically significant 

1. This appendix reports findings considered for the effectiveness rating and the average improvement indices in each domain. More detailed information on findings for all measures within the 
domains and the constructs that factor into the domains can be found in Appendices A3.2-A3.5. 
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Appendix A3.2 Summary of study findings included in the rating for the alphabetics domain 1 





Authors’ findings from the study 







Mean outcome 2 
(standard deviation) 3 


WWC calculations 


Outcome measure 

Study 

sample 4 

Sample size 
(students) 

Fast ForWord® Comparison 

group group 

Mean 

difference 5 
(Fast ForWord® 
- comparison) 

Statistical 

Effect significance 7 

size 6 (at a = 0.05) 

Improvement 

index 8 





Scientific Learning Corporation, 2007a 8 ’ 9 





Queensland University Inventory 
of Literacy (QUIL) 

Ages 5-14 

137 

8.49 

(2.31) 

7.93 

(2.58) 

0.56 

0.23 

ns 

+9 

Average for alphabetics (Scientific Learning Corporation, 2007a) 9 




0.23 

ns 

+9 





Beattie, 2000 10 





WJ-R ACH (Letter-Word 
Identification) 

Ages 12-17 

24 

90.99 

(21.29) 

92.08 

(13.15) 

-1.09 

-0.06 

ns 

-2 

WJ-R ACH (Word Attack) 

Ages 12-17 

24 

86.41 

(14.34) 

85.91 

(12.87) 

0.50 

0.04 

ns 

+1 

WJ-R COG (Auditory 
Processing Cluster for 
Phonemic Awareness) 

Ages 12-17 

24 

82.58 

(14.14) 

85.66 

(15.61) 

-3.08 

-0.20 

ns 

-8 

Wide Range Achievement 
Test-Third Edition (WRAT-3) 
(Spelling subtest) 

Ages 12-17 

24 

82.58 

(15.10) 

85.66 

(13.13) 

-3.08 

-0.21 

ns 

-8 

Average for alphabetics (Beattie, 2000)" 





-0.11 

ns 

-4 

Domain average for alphabetics across all studies 9 





0.06 

na 

+2 


ns = not statistically significant 
na = not applicable 

WJ-R = Woodcock-Johnson Psycho-Educational Battery-Revised 


1. This appendix reports findings considered for the effectiveness rating and the average improvement indices for the alphabetics domain. 

2. The intervention group values are the comparison group means plus the difference in means gains between the intervention and comparison groups. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

4. The Adolescent Literacy topic area reviews studies of interventions administered to students in grades 4-12 (or 9-18 years of age). For studies that include samples of students that span both 
the Adolescent Literacy (grades 4-12) and Beginning Reading (grades K-3) topic areas and cannot be disaggregated by grade level, the Adolescent Literacy topic area reviews any studies that 
include 5th-grade students or higher. For example, this appendix includes a combined sample of students aged 5-14 years (Scientific Learning Corporation, 2007a). 

5. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

6. For an explanation of the effect size calculation, see WWC Procedures and Standards Flandbook, Appendix B. 

7. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

(continued) 
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Appendix A3.2 Summary of study findings included in the rating for the alphabetics domain 1 (continued) 


8. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting favorable results for the intervention group. 

9. Results for the early elementary school students (in 3rd grade or below) in this study are traditionally considered under the Beginning Reading topic area reviews; however, because there was 
no separate analysis for students in 3rd grade or below (grades covered by the Beginning Reading topic area) and 4th grade and above (areas covered by the Adolescent Literacy topic area), we 
report on the total sample of students here. 

10. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 
sons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and 
Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of Beattie (2000), a correction for multiple 
comparisons was needed, so the significance level may differ from that reported in the original study. In the case of Scientific Learning Corporation (2007a), no corrections for clustering or 
multiple comparisons were needed. 

11. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated 
from the average effect sizes. 
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Appendix A3.3 Summary of study findings included in the rating for the reading fluency domain 1 





Author’s findings from the study 







Mean outcome 2 
(standard deviation) 3 


WWC calculations 


Outcome measure 

Study 

sample 

Sample size 
(students) 

Fast ForWord® Comparison 

group group 

Mean 

difference 4 
(Fast ForWord® 
- comparison) 

Statistical 

Effect significance 6 

size 5 (at a = 0.05) 

Improvement 

index 7 


Beattie, 2000 8 


Gray Oral Reading Test-Third Ages 12-17 

Edition (G0RT-3) 

24 

87.39 

(16.47) 

79.50 

(17.74) 

7.89 

0.44 

ns 

+17 

Average for reading fluency (Beattie, 2000) 9 





0.44 

ns 

+17 


ns = not statistically significant 


1. This appendix reports findings considered for the effectiveness rating and the average improvement indices for the reading fluency domain. 

2. The intervention group values are the comparison group means plus the difference in means gains between the intervention and comparison groups. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 

had more similar outcomes. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 

The improvement index can take on values between -50 and +50, with positive numbers denoting favorable results for the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple com- 
parisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures 
and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of Beattie (2000), no corrections for 
clustering and multiple comparisons were needed. 

9. This row provides the study average, which in this instance is also the domain average. The domain improvement index is calculated from the average effect size. 
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Appendix A3.4 Summary of study findings included in the rating for the comprehension domain 1 





Authors’ findings from the study 







Mean outcome 2 
(standard deviation) 3 


WWC calculations 


Outcome measure 

Study 

sample 4 

Sample size 
(students) 

Fast ForWord® Comparison 

group group 

Mean 

difference 5 
(Fast ForWord® 
- comparison) 

Statistical 

Effect significance 7 

size 6 (at a = 0.05) 

Improvement 

index 8 




Beattie, 2000 9 





WJ-R ACH Passage Ages 12-17 

Comprehension 

24 

97.17 93.25 

(13.78) (11.30) 

3.92 

0.30 

ns 

+12 

Average for comprehension (Beattie, 2000) 10 




0.30 

ns 

+12 



Borman & Benson, 2006 s 





CTBS/5 Terra Nova Reading Grade 7 

NCE Scores 

188 

36.99 34.03 

(14.11) (14.92) 

2.96 

0.20 

ns 

+8 

Average for comprehension (Borman & Benson, 2006) 9 




0.20 

ns 

+8 



Overbay & Baenen, 2002 811 





North Carolina End of Grade Grades 

4-8 

710 

154.37 155.37 

-1.00 

-0.14 

ns 

-6 

Average for comprehension (Overbay & Baenen, 2002) 9 




-0.14 

ns 

-6 



Scientific Learning Corporation, 2004a 8 





Ohio Proficiency Test, Grade 4 

Reading Test 

91 

210.60 205.10 

(16.65) (16.97) 

5.50 

0.32 

ns 

+13 

Average for comprehension (Scientific Learning Corporation, 2004a) 9 



0.32 

ns 

+13 



Scientific Learning Corporation, 2004b 812 





Gates— MacGinitie Reading Test Grades 2-8 

293 

30.39 25.00 

(14.37) (10.60) 

5.39 

0.39 

Statistically 

significant 

+15 

Average for comprehension (Scientific Learning Corporation, 2004b) 9 



0.39 

Statistically 

significant 

+15 



Scientific Learning Corporation, 2007b 813 





Measures of Academic Grades 4-5 

Progress, Reading Test 

70 

34.90 30.50 

(32.00) (30.10) 

4.40 

0.14 

ns 

+6 

Average for comprehension (Scientific Learning Corporation, 2007b) 9 



0.14 

ns 

+6 

Domain average for comprehension across all studies 9 




0.20 

na 

+8 


(continued) 
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Appendix A3.4 Summary of study findings included in the rating for the comprehension domain 1 (continued) 


ns = not statistically significant 

na = not applicable 

WJ-RACH = Woodcock-Johnson Psycho-Educational Battery-Revised, Tests of Achievement 

CTBS/5 = Comprehensive Test of Basic Skills 

NCE = Normal Curve Equivalent 

1. This appendix reports findings considered for the effectiveness rating and the average improvement indices for the comprehension domain. Subgroup findings from the same studies are not 
included in these ratings, but are reported in Appendix A4. 

2. The intervention and control group values for Scientific Learning Corporation (2007b) are the ANCOVA adjusted mean values calculated using pretest scores as the covariates. For all other stud- 
ies in this domain, the intervention group values are the comparison group means plus the difference in means gains between the intervention and comparison groups. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

4. The Adolescent Literacy topic area reviews studies of interventions administered to students in grades 4-12 (or 9-18 years of age). For studies that include samples of students that span both 
the Adolescent Literacy (grades 4-12) and Beginning Reading (grades K-3) topic areas and cannot be disaggregated by grade level, the Adolescent Literacy topic area reviews any studies that 
include 5th-grade students or higher. For example, this appendix includes a combined sample of students from grades 2-8 (Scientific Learning Corporation, 2004b). 

5. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

6. For an explanation of the effect size calculation, see WWC Procedures and Standards Flandbook, Appendix B. 

7. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

8. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting favorable results for the intervention group. 

9. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple 
comparisons were not done for findings not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas 
the WWC used to calculate the statistical significance, see WWC Procedures and Standards Flandbook, Appendix C. For the Fast ForWord m studies summarized here, no corrections for cluster- 
ing or multiple comparisons were needed. 

10. This row provides the study average, which in this instance is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The 
domain improvement index is calculated from the average effect size. 

11. This study reported the mean values for the outcome measure, not the standard deviations. The effect size for each grade was calculated through the F-statistics from the one way ANOVA 
reported in the study. The average effect size reported here is based on effect sizes that have been weighted by the sample size for each grade. 

12. The means and standard deviations were aggregated across two intervention groups. 

13. This study separately reported results for students in grades 3 and below and for students in grades 4 and above, along with aggregated results across all of the grade levels. Results for the 
second- and third-grade students in this study will be considered under the Beginning Reading topic area reviews. 
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Appendix A3.5 Summary of study findings included in the rating for the general literacy achievement domain 1 





Authors’ findings from the study 







Mean outcome 2 
(standard deviation) 3 


WWC calculations 


Outcome measure 

Study 

sample 4 

Sample size 
(students) 

Fast ForWord® Comparison 

group group 

Mean 

difference 5 
(Fast ForWord® 
- comparison) 

Statistical 

Effect significance 7 

size 6 (at a = 0.05) 

Improvement 

index 8 


Success for All Assessment 

Grades 3-6 

373 

Rouse & Krueger, 2004 8 

4.06 4.03 

(1.40) (1.33) 

0.03 

0.02 

ns 

+1 

CELF-3, Receptive Language 

Grades 3-6 

86 

31.70 

(18.43) 

31.01 

(16.59) 

0.69 

0.04 

ns 

+2 

State Standardized Reading Test 

Grades 3-6 

454 

44.18 

(24.79) 

43.03 

(24.01) 

1.15 

0.05 

ns 

+2 

Average for general literacy achievement (Rouse & Krueger, 2004) 9 




0.04 

ns 

+1 




Scientific Learning Corporation, 2007a 8 9 




CELF-4, Receptive Language 

Ages 5-14 

137 

91.00 

88.40 

2.60 

0.19 

ns 

+8 




(12.40) 

(14.12) 





CELF-4, Expressive Language 

Ages 5-14 

137 

88.00 

85.00 

3.00 

0.23 

ns 

+9 




(12.40) 

(13.29) 





Average for general literacy achievement (Scientific Learning Corporation, 2007a) 9 



0.21 

ns 

+8 




Beattie, 2000 8 





CELF-3, Receptive Language 

Ages 12-17 

24 

86.08 

86.83 

-0.75 

-0.03 

ns 

-1 




(21.11) 

(22.74) 





Average for general literacy achievement (Beattie, 2000) 9 




-0.03 

ns 

-1 




Borman & Benson, 2006 10 





CTBS/5 Terra Nova Language 

Grade 7 

188 

40.52 

40.14 

0.38 

0.03 

ns 

+1 

NCE Scores 



(11.22) 

(11.59) 





Average for general literacy achievement (Borman & Benson, 2006) 11 




0.03 

ns 

+1 




Scientific Learning Corporation, 2007b 8 ’ 12 




Measures of Academic 

Grades 4-5 

70 

31.10 

26.80 

4.30 

0.16 

ns 

+6 

Progress, Language Test 



(27.90) 

(26.60) 





Average for general literacy achievement (Scientific Learning Corporation, 2007b) 9 



0.16 

ns 

+6 

Domain average for general literacy achievement across all studies 9 




0.08 

na 

+3 


(continued) 
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Appendix A3.5 Summary of study findings included in the rating for the general literacy achievement domain 1 (continued) 


ns = not statistically significant 

na = not applicable 

CELF-3 = Clinical Evaluation of Language Fundamentals-Third Edition 

CELF-4 = Clinical Evaluation of Language Fundamentals-Fourth Edition 

CTBS/5 = Comprehensive Test of Basic Skills 

NCE = Normal Curve Equivalent 

1. This appendix reports findings considered for the effectiveness rating and the average improvement indices for the general literacy achievement domain. 

2. The intervention group values are the comparison group means plus the difference in means gains between the intervention and comparison groups tested immediately after the intervention for 
Beattie (2000) and Scientific Learning Corporation (2007a). The intervention group values are the comparison group means plus the regression-adjusted impacts for Borman and Benson (2006) 
and Rouse and Krueger (2004). The intervention and control group values for Scientific Learning Corporation (2007b) are the ANCOVA adjusted mean values calculated using pretest scores as 
the covariates. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

4. The Adolescent Literacy topic area reviews studies of interventions administered to students in grades 4-12 (or 9-18 years of age). For studies that include samples of students that span both 
the Adolescent Literacy (grades 4-12) and Beginning Reading (grades K-3) topic areas and cannot be disaggregated by grade level, the Adolescent Literacy topic area reviews any studies that 
include 5th-grade students or higher. For example, this appendix includes a combined sample of students from grades 3-6 (Rouse & Krueger, 2004) and students aged 5-14 years (Scientific 
Learning Corporation, 2007a). 

5. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

6. For an explanation of the effect size calculation, see WWC Procedures and Standards Flandbook, Appendix B. 

7. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

8. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting favorable results for the intervention group. 

9. Results for the early elementary school students (in grades 3 and below) in this study are traditionally considered under the Beginning Reading topic area reviews; however, because there was 
no separate analysis for students in 3rd grade or below (grades covered by the Beginning Reading topic area) and 4th grade and above (areas covered by the Adolescent Literacy topic area), we 
report on the total sample of students here. 

10. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 
sons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and 
Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. For all Fast ForWord ® studies summarized here, except 
Beattie (2000) and Borman & Benson (2006), a correction for multiple comparisons was needed, so the significance levels may differ from those reported in the original study. 

11. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated 
from the average effect sizes. 

12. This study separately reported results for students in grades 3 and below and for students in grades 4 and above, along with aggregated results across all of the grade levels. Results for the 
2nd- and 3rd-grade students in this study will be considered under the Beginning Reading topic area reviews. 
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Appendix A4 Summary of subgroup findings for the comprehension domain 1 


Authors’ findings from the study 


Mean outcome 2 

(standard deviation) 3 WWC calculations 

Mean 

difference 5 Statistical 


Outcome measure 

Study 

sample 4 

Sample size 
(students) 

Fast ForWord® 
group 

Comparison 

group 

(Fast ForWord® 
- comparison) 

Effect 

size 6 

significance 7 
(at o = 0.05) 

Improvement 

index 8 





Overbay & Baenen, 2002 9 





North Carolina 
End of Grade Test 

Grade 4 

114 

148.39 

150.90 

-2.51 

-0.35 

ns 

-14 

North Carolina 
End of Grade Test 

Grade 5 

148 

156.07 

155.76 

0.31 

0.06 

ns 

+2 

North Carolina 
End of Grade Test 

Grade 6 

78 

149.80 

151.59 

-1.79 

-0.23 

ns 

-9 

North Carolina 
End of Grade Test 

Grade 7 

224 

155.70 

156.86 

-1.16 

-0.18 

ns 

-7 

North Carolina 
End of Grade Test 

Grade 8 

146 

157.70 

158.18 

-0.48 

-0.08 

ns 

-3 




Scientific Learning Corporation, 2004b 10 ' 11 




Gates— MacGinitie 
Reading Test 

Group 1 vs. control; 
grades 2-8 

162 

30.70 

(13.90) 

25.00 

(10.60) 

5.70 

0.43 

Statistically 

significant 

+17 

Gates— MacGinitie 
Reading Test 

Group 2 vs. control; 
grades 2-8 

168 

30.10 

(14.80) 

25.00 

(10.60) 

5.10 

0.36 

ns 

+14 

Gates— MacGinitie 
Reading Test 

Group 1 vs. control; 
grade 4 

56 

26.90 

(12.80) 

23.20 

(10.20) 

3.70 

0.31 

ns 

+12 

Gates— MacGinitie 
Reading Test 

Group 2 vs. control; 
grade 4 

67 

27.00 

(15.20) 

23.20 

(10.20) 

3.80 

0.28 

ns 

+11 

Gates— MacGinitie 
Reading Test 

Group 1 vs. control; 
grade 5 

103 

35.40 

(14.10) 

30.60 

(9.10) 

4.80 

0.34 

ns 

+14 

Gates— MacGinitie 
Reading Test 

Group 2 vs. control; 
grade 5 

83 

34.90 

(13.40) 

30.60 

(9.10) 

4.30 

0.33 

ns 

+13 


(continued) 
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Appendix A4 Summary of subgroup findings for the comprehension domain 1 (continued) 


ns = not statistically significant 

1. This appendix presents subgroup findings for measures that fall in the comprehension domain. Total group scores were used for rating purposes and are presented in Appendix A3.4. 

2. The intervention group values are the comparison group means plus the difference in means gains between the intervention and comparison groups. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 

had more similar outcomes. 

4. The Adolescent Literacy topic area reviews studies of interventions administered to students in grades 4-12 (or 9-18 years of age). For studies that include samples of students that span both 
the Adolescent Literacy (grades 4-12) and Beginning Reading (grades K-3) topic areas and cannot be disaggregated by grade level, the Adolescent Literacy topic area reviews any studies that 
include 5th-grade students or higher. For example, this appendix includes a combined sample of students from grades 2-8 (Scientific Learning Corporation, 2004b). 

5. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

6. For an explanation of the effect size calculation, see WWC Procedures and Standards Flandbook, Appendix B. 

7. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

8. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

9. This study reported the mean values only for the outcome measure, not the standard deviations. The effect size was calculated through the F-statistics from the one way ANOVA reported in 

the study. 

10. Treatment group 1 received the intervention from September to November, and treatment group 2 received the intervention from December to February. 

11. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple 
comparisons were not done for findings not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas 
the WWC used to calculate the statistical significance, see WWC Procedures and Standards Flandbook, Appendix C. For the Fast ForWord® studies summarized here, no corrections for cluster- 
ing were needed. 
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Appendix A5.1 Fast ForWord rating for the alphabetics domain 

The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of alphabetics, the WWC rated Fast ForWord ® as having no discernible effects for adolescent learners. It did not meet the criteria for posi- 
tive effects, potentially positive effects, mixed effects, potentially negative effects, or negative effects because no studies showed statistically significant or substan- 
tively important effects, either positive or negative. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: No studies showing a statistically significant or substantively important effect, either positive or negative. 

Met. No studies showed a statistically significant or substantively important effect, either positive or negative. Two studies showed indeterminate effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed a statistically significant positive effect. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies showed a statistically significant or substantively important negative effect. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Not met. No studies showed a statistically significant or substantively important positive effect. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing 
indeterminate effects than showing statistically significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important negative effect. Two studies showed indeterminate effects, and no 
studies showed a statistically significant or substantively important positive effect. 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No studies showed a statistically significant or substantively important positive effect, and no studies showed a statistically significant or 
substantively important negative effect. 

OR 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No studies showed a statistically significant or substantively important effect. 

(continued) 
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Appendix A5.1 Fast ForWord rating for the alphabetics domain (continued) 


Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: One study showing a statistically significant or substantively important negative effect and no studies showing a statistically significant or substantively 
important positive effect. 

Not met. No studies showed a statistically significant or substantively important effect, either positive or negative. 

OR 

• Criterion 2: Two or more studies showing statistically significant or substantively important negative effects, at least one study showing a statistically significant 
or substantively important positive effect, and more studies showing statistically significant or substantively important negative effects than showing statistically 
significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important effect, either positive or negative. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed a statistically significant negative effect. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. No studies showed a statistically significant or substantively important positive effect. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. For a complete description, see the WWC Procedures and Standards Handbook, Appendix E. 
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Appendix A5.2 Fast ForWord rating for the reading fluency domain 

The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of reading fluency, the WWC rated Fast ForWord ® as having potentially positive effects for adolescent learners. It did not meet the criteria 
for positive effects because no studies showed statistically significant positive effects. The remaining ratings (mixed effects, no discernible effects, potentially negative 
effects, or negative effects) were not considered, as Fast ForWord ® was assigned the highest applicable rating. 


Rating received 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Met. One study showed a substantively important positive effect. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indetermi- 
nate effects than showing statistically significant or substantively important positive effects. 

Met. No studies showed a statistically significant or substantively important negative effect, no studies showed indeterminate effects, and one 
study showed substantively important positive effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed statistically significant positive effects. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies showed a statistically significant or substantively important negative effect. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. For a complete description, see the WWC Procedures and Standards Handbook, Appendix E. 
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Appendix A5.3 Fast ForWord^ rating for the comprehension domain 

The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of comprehension, the WWC rated Fast ForWord® as having potentially positive effects for adolescent learners. It did not meet the criteria 
for positive effects because only one study showed statistically significant positive effects. The remaining ratings (mixed effects, no discernible effects, potentially nega- 
tive effects, or negative effects) were not considered, as Fast ForWord® was assigned the highest applicable rating. 


Rating received 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Met. One study showed a statistically significant positive effect, and two studies showed substantively important positive effects. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indetermi- 
nate effects than showing statistically significant or substantively important positive effects. 

Met. No studies showed a statistically significant or substantively important negative effect. Three studies showed indeterminate effects, one study 
showed a statistically significant positive effect, and two studies showed substantively important positive effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. Only one study showed a statistically significant positive effect. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies showed a statistically significant or substantively important negative effect. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. For a complete description, see the WWC Procedures and Standards Handbook, Appendix E. 
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Appendix A5.4 Fast ForWord rating for the general literacy achievement domain 

The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of general literacy achievement, the WWC rated Fast ForWord®as having no discernible effects for adolescent learners. It did not meet the 
criteria for positive effects, potentially positive effects, mixed effects, potentially negative effects, or negative effects because no studies showed statistically significant 
or substantively important effects, either positive or negative. 


Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: No studies showing a statistically significant or substantively important effect, either positive or negative. 

Met. No studies showed a statistically significant or substantively important effect, either positive or negative. Five studies showed indeterminate effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed a statistically significant positive effect. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies showed a statistically significant or substantively important negative effect. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Not met. No studies showed a statistically significant or substantively important positive effect. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing 
indeterminate effects than showing statistically significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important negative effect. Five studies showed indeterminate effects, and no 
studies showed a statistically significant or substantively important positive effect. 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No studies showed a statistically significant or substantively important negative effect. 

OR 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No studies showed a statistically significant or substantively important effect. 

(continued) 
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Appendix A5.4 Fast ForWord rating for the general literacy achievement domain (continued) 


Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: One study showing a statistically significant or substantively important negative effect and no studies showing a statistically significant or substantively 
important positive effect. 

Not met. No studies showed a statistically significant or substantively important effect, either positive or negative. 

OR 

• Criterion 2: Two or more studies showing statistically significant or substantively important negative effects, at least one study showing a statistically significant 
or substantively important positive effect, and more studies showing statistically significant or substantively important negative effects than showing statistically 
significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important effect, either positive or negative. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed a statistically significant negative effect. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. No studies showed a statistically significant or substantively important positive effect. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. For a complete description, see the WWC Procedures and Standards Handbook, Appendix E. 
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Appendix A6 Extent of evidence by domain 


Outcome domain 

Number of studies 

Schools 

Sample size 

Students 

Extent of evidence 1 

Alphabetics 

2 

7 

161 

Small 

Reading fluency 

1 

3 

24 

Small 

Comprehension 

6 

>42 2 

1,376 

Medium to large 

General literacy achievement 

5 

20 

873 3 

Medium to large 


1. A rating of “medium to large” requires at least two studies and two schools across studies in one domain and a total sample size across studies of at least 350 students or 14 classrooms. Other- 
wise, the rating is “small.” For more details on the extent of evidence categorization, see the WWC Procedures and Standards Handbook, Appendix G. 

2. The number of control schools in Overbay and Baenen (2002) is unknown. 

3. For Rouse and Krueger (2004), we counted the number of students as 454, which is from the state assessment. The actual number of students might be higher, as we do not know to what extent 
the number of students from the three outcome measures overlapped. 
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